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(54) Computer system having alphanumeric keyboard access to objects in graphical user 
interface 



(57) An electronic digital computer system (100) 
includes a graphical user interface (120) (GUI) for dis- 
playing graphical objects (52, 56, 58, 62, 64, 68, 72, 74, 
76) such as icon buttons (50), scrolling text lists (58), 
and text entry fields (72) on a monitor (106). A keyboard 
(1 14) includes a label key (1 14b) such as a conventional 
alternate (Alt) key (114b) which, when depressed, 
causes alphanumeric characters (A-V, 1-7) to be super- 
imposed on the graphical objects (52, 56, 58, 62, 64, 68, 
72, 74, 76). Depression of an alphanumericic key (1 1 4c) 
on the keyboard (114) corresponding to the alphanu- 
meric character (A-V, 1-7) superimposed on a selected 
object (52, 56. 58. 62, 64. 68. 72, 74, 76) causes an 
associated operation to be launched. The GUI includes 
an object table (122a) having an entry for each graphi- 



cal object (52. 56, 58, 62, 64. 68. 72, 74, 76). Each entry 
includes an object name, corresponding alphanumeric 
character (A-V, 1-7), object type (e.g. icon button (56), 
text input field (72)), and object display position. A dis- 
play generator (122) accesses the object table (122a) 
using the object names as inputs, obtains the alphanu- 
meric characters (A-V. 1-7) as outputs, and superim- 
poses and displays the alphanumeric characters (A-V, 
1-7) on the objects (52, 56, 58, 62, 64, 68. 72, 74, 76). 
The alphanumeric characters (A-V, 1 -7) are displayed at 
positions that are offset from the object display positions 
by predetermined distances and directions correspond- 
ing to the object types. 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention s 

The present invention generally relates to the art of 
electronic digital computers having graphical object dis- 
plays, and more specifically to a computer system hav- 
ing alphanumeric keyboard access to objects in a 10 
Graphical User Interface. 

Description of the Related Art 

A Graphical User Interface (GUI) is a convenient 15 
mechanism for enabling a user to interface with an elec- 
tronic digital computer. A GUI displays various graphical 
interface objects on a monitor screen which can be 
selected using a pointing device such as a mouse or a 
trackball . The user moves the pointing device to position 20 
an arrow or other cursor icon over an object to be 
selected, and clicks a button on the pointing device to 
make the selection. In response, an operation (proce- 
dure, application, etc.) which is represented by the 
object is launched. 25 

The graphical interface objects typically include but- 
tons with iconic labels, scrolling text lists, and text entry 
fields that are very convenient to access with a pointing 
device. However, pointing devices are not usable by 
some persons, including those suffering from afflictions 30 
of the hand or wrist such as carpal tunnel syndrome. 
This latter group of users are unable to position a 
mouse and simultaneously click a button on the mouse 
due to severe wrist pain. 

Pointing devices are also not usable by disabled 35 
persons such as quadriplegics who are, however, able 
to depress individual keys on a keyboard using sticks 
held in their mouths. 

Other users, although not suffering from any disa- 
bility, simply do not like to use pointing devices, and 40 
some power users feel that pointing devices are too 
slow. 

It is therefore desirable to provide a mechanism by 
which users such as described above can access a 
graphical interface object of a GUI directly from a con- 45 
ventional alphanumeric keyboard. Various mechanisms 
have been proposed to accomplish this purpose in the 
prior art, such as illustrated in FIG. 1. 

FIG. 1 illustrates exemplary keyboard accessible 
graphical objects of a GU1 1 0 in the form of an electronic so 
mail (e-mail) system. It will be noted that this drawing is 
not intended to illustrate an actual application, but is a 
composite diagram for presenting various types of prior 
art GUI displays. 

The GU1 1 0 includes a menu bar 1 2 having a "mne- ss 
monic character" graphical display including mnemonic 
text labels that enable a user to, for example, read mail 
(READ), reply to a received message (REPLY), forward 
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a received message (FORWARD), compose a new 
message (COMPOSE), edit a new message (EDIT), 
send a new message (SEND), or save a message to 
disk (SAVE). 

More specifically, one letter of each text label in the 
menu bar 12 is underlined. The user can select an 
option, for example reading mail (READ), by depressing 
a label key such as a conventional alternate (Alt) key 
while simultaneously depressing the underlined letter 
"R" in the word "READ". 

FIG. 1 further illustrates a "keyboard accelerator" 
box 14 including a menu having the same options as 
presented in the menu bar 12. In this case, each menu 
item includes a word representing the option, and a 
combination of keystrokes or "hot key" combination that 
can be used to immediately launch the corresponding 
operation. In this exemplary case also, the read option 
can be activated by simultaneously depressing the Alt 
key and the R key. 

Although usable, these prior art mechanisms for 
accessing graphical interface objects using an alphanu- 
meric keyboard are undesirable in that a separate menu 
box including a text list of the available options is 
required in addition to any graphical icons for the same 
items. These methods are not usable at all for GUI dis- 
plays which consist entirely of icon buttons and text 
input fields without text labels as will be described 
below. 

FIG. 1 further illustrates an icon button panel 16 
which consists of graphical icon buttons for selecting the 
same options as in the menu bar 12. A "scrolling win- 
dow" 18 is also shown, including a column of text items 
20 such as received messages, up and down arrows 22 
and 24, and a fixed rectangular frame 26 or highlight 
block which indicates a current text item. 

The scrolling window 18 is conventionally operated 
by using a mouse or other pointing device to position an 
arrow 28 or other cursor icon on the arrow 22 or 24, and 
clicking or holding down the mouse button to cause the 
list of text items 20 to scroll up or down until the desired 
text item is positioned in the frame 26. 

The user then positions the arrow 28 on the text 
item in the frame using the mouse, and clicks the mouse 
button to select the item. Alternatively, the arrow 28 can 
be positioned on a text item which is not in the frame 26, 
and the mouse button clicked to select the item. Finally, 
an Enter or other key on the keyboard can be depressed 
to activate a text item which is positioned in the frame 
26. 

Further illustrated is a sort bar 30 having buttons 
which can be selected using the mouse to cause the 
text entries to be sorted by date, sender, recipient, or 
subject. 

FIG. 1 further illustrates a "text entry" box 32 which 
enables a user to compose a new message using an 
alphanumeric keyboard. The box 32 includes a text 
entry field 34 which displays the actual text being 
entered. Two additional text entry fields 36 and 38 are 
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provided for entering the name of the addressee and 
the subject of the message. 

The field 34, 36 or 38 into which text is desired to be 
entered is selected by positioning the arrow 28 in the 
corresponding field using the mouse, and clicking the s 
mouse button. As shown, an insertion cursor 40 in the 
form of a vertical "I-beam" is displayed after the letter 
"H" in the subject field 38, indicating that the subject 
field 38 has been selected for text entry, and that the 
text entry position is at the end of the existing text in the 10 
field 38. 

FIG. 1 further illustrates a menu box 42 including 
the same options as in the menu bar 12. This selection 
mechanism uses a rectangular location cursor 45 and a 
selection bar 44 to designate a menu option. The selec- 15 
tion bar 44 is moved by repeatedly depressing an arrow 
key on a keyboard. Each keypress advances the cursor 
to the next menu option. When the cursor 44 has been 
moved to the desired option, a space key or other key is 
depressed to activate the option. 20 

The location cursor arrangement can be expanded 
globally to enable selection of all of the illustrated 
menus, icon buttons and text input fields from the key- 
board. In such a case, the user repeatedly depresses 
another key, such as the Tabulation (Tab) key, to step a 25 
cursor 45 through the major blocks of the display 1 0. For 
example, repeated depression of the Tab key will move 
the cursor from the menu bar 12 to the keyboard accel- 
erator box 14, to the first button in the icon button panel 
16, to the second button in the panel 16, etc. 30 

The cursor 45 is shown as being positioned over 
the menu box 42, and can be in the form of a rectangu- 
lar box, a different color block, inverse video, etc. After 
the desired major block is selected using the Tab key, in 
this case the menu box 32, the keyboard arrow keys are 35 
used to move the cursor 44 within the block, and the 
space key is depressed to select the desired option 
within the block as described above. 

Although enabling access to alt of the graphical 
objects of the display 10 from an alphanumeric key- 40 
board, the global location cursor arrangement is unde- 
sirable in that the tab key may have to be depressed 
many times to move the cursor to a selected item. 

In summary, the scrolling list display and text entry 
field selection mechanism as illustrated in FIG. 1 are not 45 
efficiently accessible from a keyboard. There is a need 
to provide this capability which will especially enable the 
special types of users described above to access 
graphical interface objects in accordance with their 
requirements. so 

SUMMARY OF THE INVENTION 

The present invention overcomes the drawbacks of 
the prior art, and provides the heretofore needed ability 55 
to efficiently access graphical interface objects using a 
conventional alphanumeric keyboard. This includes 
types of objects such as graphical icon buttons and 



scrolling text lists that could be accessed only with con- 
siderable difficulty in the prior art. 

In accordance with the present invention, an elec- 
tronic digital computer system includes a Graphical 
User Interface (GUI) which utilizes a unique method for 
displaying and selecting graphical objects such as icon 
buttons, scrolling lists, and text entry fields on a monitor. 

A keyboard includes a label key such as a conven- 
tional alternate (Alt) key which, when depressed, 
causes alphanumeric characters to be superimposed 
on the graphical objects. Depression of an alphanu- 
meric key on the keyboard corresponding to the alpha- 
numeric character superimposed on a selected object 
causes an associated operation (procedure, applica- 
tion, etc.) to be launched. 

The GUI includes an object table having an entry 
for each graphical object. Each entry includes an object 
name, corresponding alphanumeric character, object 
type (e.g. icon button, text input field), and object display 
position. A display generator accesses the object table 
using the object names as inputs, obtains the alphanu- 
meric characters as outputs, and superimposes and 
displays the alphanumeric characters on the objects. 

The alphanumeric characters are displayed at posi- 
tions which are offset from the object display positions 
by predetermined distances and directions correspond- 
ing to the object types. 

The present invention improves over prior art mne- 
monic character, keyboard accelerator and location cur- 
sor arrangements in that it does not require a separate 
text menu, or require that objects have text labels. The 
present arrangement is much faster than a location cur- 
sor system in that only one keystroke is required to 
select a graphical object. 

The arrangement of the invention is especially 
desirable for persons suffering from afflictions such as 
carpal tunnel syndrome, because any graphical object 
can be selected using simple key presses. The inven- 
tion also enables quadriplegics to access graphical user 
interfaces from a keyboard using a single stick 

These and other features and advantages of the 
present invention will be apparent to those skilled in the 
art from the following detailed description, taken 
together with the accompanying drawings, in which like 
reference numerals refer to like parts. 

PESCRIPTION OF THE DRAWINGS 

FIG. 1 is a composite diagram illustrating prior art 
methods of accessing graphical interface objects 
on a computer display using an alphanumeric key- 
board; 

FIG. 2 is a diagram illustrating the appearance of a 
graphical object display in accordance with the 
present invention prior to activation for keyboard 
object access; 

FIG. 3 is similar to FIG. 2, but illustrates the display 
after activation; 
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FIG. 4 is similar to FIG. 3, but illustrates activation 
of a second level of graphical objects for keyboard 
access; 

FIG. 5 is a simplified block diagram illustrating a 
computer system according to the present inven- 5 
tion; 

FIG. 6 is a block diagram illustrating a Graphical 
User Interface (GUI) of the computer system of 
FIG. 5; 

FIG. 7 is a block diagram illustrating a Random w 
Access Memory (RAM) of the computer system of 
FIG. 5 which stores the GUI of FIG. 6; 
FIG. 8 is a flowchart illustrating a main event loop of 
an application program; 

FIG. 9 is a flowchart illustrating a procedure which is 
checks for depression of a label key; 
FIG. 10 is a flowchart illustrating a procedure which 
displays alphanumeric characters superimposed 
on graphical objects in response to depression of 
the label key; 20 
FIG. 1 1 is a flowchart illustrating a display proce- 
dure; and 

FIG. 12 is a flowchart illustrating a routine for 
launching a procedure corresponding to a selected 
graphical object. 25 

DETAILED DESCRIPTION OF THE INVENTION 

An exemplary Graphic User Interface (GUI) display 
50 according to the present invention is illustrated in 30 
FIG. 2. The drawing illustrates an electronic mail (e- 
mail) system, although the invention is not limited to any 
particular application. 

The display 50 includes a menu box 52 with a plu- 
rality of text labels corresponding to options including 35 
INBOX, ADDRESS BOOK, etc. as illustrated. These 
options are normally accessed by positioning an arrow 
54 or other cursor icon over the text label of the desired 
option using a pointing device such as a mouse, and 
clicking a button on the mouse to select the option. 40 

FIG. 2 further illustrates an icon button panel 56 
which consists of graphical icon buttons that enable a 
user to, viewed from left to right, read mail, reply to a 
received message, forward a received message, com- 
pose a new message, edit a new message, send a new 45 
message, or save a message to disk. These buttons are 
normally accessed by moving the arrow 54 over the but- 
ton corresponding to the selected option, and clicking 
the mouse button. 

A "scrolling window" 58 is also shown, including a so 
column of text items 60 such as received messages, up 
and down arrows 62 and 64, and a rectangular frame 66 
or highlight block which acts as a cursor to indicate a 
current text item. 

The scrolling window 58 is conventionally operated 55 
by using a mouse or other pointing device to position the 
arrow 54 over the arrow 62 or 64, and clicking or holding 
down the mouse button to cause the frame 66 to move 



up or down on the display. Further illustrated is a sort 
bar 68 having buttons which can be selected using the 
mouse to cause the text entries to be sorted by date, 
sender, recipient, or subject. 

FIG. 2 further illustrates a "text entry" box 70 which 
enables a user to compose a new message using an 
alphanumeric keyboard. The box 70 includes a text 
entry field 72 which displays the actual text being 
entered. Two additional text entry fields 74 and 76 are 
provided for entering the name of the addressee and 
the subject of the message. 

The field 72, 74 or 76 into which text is desired to be 
entered is conventionally selected by positioning the 
arrow 54 in the corresponding field using the mouse, 
and clicking the mouse button. As shown, an insertion 
cursor 78 is displayed after the letter "H" in the subject 
field 76 indicating that the subject field 76 has been 
selected for text entry, and that the current text entry 
position is at the end of the field. 

It will be noted that a GUI embodying the present 
invention preferably retains the capability of conven- 
tional access using a pointing device such as a mouse, 
and that this method of access can be used instead of 
or in combination with the keyboard access provided by 
the invention as desired by a particular user. 

In accordance with the present invention as illus- 
trated in FIG. 3, all of the graphical interface objects in 
the display 50 can be accessed efficiently using a con- 
ventional alphanumeric keyboard, without the need of 
having a pointing device connected to the computer. 

As will be described in detail below, the display 50 
of FIG. 2 is transformed into a display 50' as illustrated 
in FIG. 3 by depressing a "label key" on an alphanu- 
meric keyboard. The label key is preferably the alternate 
(Alt) key, although the invention is not so limited. 

It is within the scope of the invention to use another 
key, such as the control (Ctl) key as the label key. 
Another option is to use a combination of keys as the 
"label key". For example, the display 50* can be pro- 
duced by simultaneously depressing the Ctl and Alt 
keys. 

The display 50' differs from the display 50 in that 
alphanumeric labels or "activation characters" are 
superimposed on each graphical interface object. As 
shown in the drawing, activation characters "A" to "F" 
are superimposed on the text labels in the menu box 52, 
activation characters "G" to "M" are superimposed on 
the icon buttons in the icon button panel 56, and activa- 
tion characters "N" to "Q" are superimposed on the sort 
fields in the sort bar 68. 

In addition, the character "R" is superimposed on 
the scrolling window 58, the character "S" is superim- 
posed on the area including the up and down arrows 62 
and 64, and the characters T, "U" and "V" are super- 
imposed on the addressee, subject and message fields 
of the text entry box 70. 

It will be noted that the term "superimposed on" as 
used in the context of the present invention does not 
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require that the alphanumeric characters be displayed 
directly over and obscure portions of underlying graphi- 
cal objects. The characters can be displayed closely 
adjacent to the corresponding objects as in the illus- 
trated menu box 52 within the meaning of the term as 5 
used herein. 

As is further shown in FIG. 3, the alphanumeric 
characters are illustrated as being enclosed in "frames" 
as indicated at 79. The frames 79 have oval shapes, 
with their height being larger than their width. However, 10 
the shape of the frames 79, or their provision, is not lim- 
ited within the scope of the invention. 

It is further preferable to increase the legibility of the 
display by obscuring all visible matter within the frames 
79 except for the characters. For example, the frames 15 
79 themselves can be black, the area within the frames 
79 can be white, and the characters can be black. Thus, 
the legibility of the characters will not be reduced by any 
visible matter in a graphical object over which a charac- 
ter is superimposed. 20 

After the display 50' of FIG. 3 has been activated by 
depressing the label key or keys, a desired graphical 
object is selected by depressing a key on the keyboard 
corresponding to the alphanumeric character superim- 
posed on the object. For example, the text entry field 70 25 
can be selected by depressing the "V" key on the key- 
board. In this case, the field 70 is activated for compos- 
ing the text of a message, and a cursor 80 is positioned 
at the end of any existing text. If the field does not con- 
tain any text, the cursor 80 is positioned at the first text 30 
entry position. 

The other graphical objects are selected in the 
same manner. Selection of an object causes an associ- 
ated operation to be performed. The operation can be 
launching a procedure or application, or performing any 35 
other suitable function. For example, depression of the 
"S" key activates the scroll function of the window 58. 
Rather than scrolling the text items 60 by using a mouse 
to select one of the arrows 62 and 64, scrolling is per- 
formed using up and down arrows on the keyboard. 40 

The label key operation can be implemented in sev- 
eral ways within the scope of the invention. One alterna- 
tive is to have the display 50' be generated as long as 
the label key (which will hereinafter assumed to be the 
Alt key) is depressed, and the operation corresponding 45 
to the selected graphical object launched upon depres- 
sion of the associated alphanumeric key. This requires 
that the Alt key and the alphanumeric key be depressed 
simultaneously. 

Another alternative is to use a "sticky key" arrange- so 
ment, in which the display 50' will be generated upon 
depression of the Alt key, and will continue to be dis- 
played even if the Alt key is released. The desired 
graphical object is then selected and the associated 
operation launched by depressing the corresponding 55 
alphanumeric key. 

The sticky key arrangement further provides means 
to cancel the activation character display if it was 



selected erroneously, or if the user changes his or her 
mind. Preferably, the activation character display can be 
canceled and the original display 50 of FIG. 2 restored 
by depressing the label key a second time. However, the 
invention is not so limited, and any other key press or 
sequence can be provided for canceling the activation 
character display. 

In the sticky key embodiment, only one key need be 
depressed at any one time. This is especially desirable 
for quadriplegics who are only able to depress one key 
at a time using a stick. 

The present invention can be expanded into a hier- 
archical or tree structure as illustrated in FIG. 4. In this 
case, depression of the "D" key which corresponds to 
the text object "FOLDERS" causes a submenu 82 to be 
generated. The submenu 82 includes a list of directories 
or folders in the user's e-mail system such as "MAN- 
AGEMENT", "VENDORS", etc., with the alphanumeric 
characters "1" to "7" being superimposed on (in this 
case displayed closely adjacent to) the corresponding 
text objects. 

Depressing the "7" key on the keyboard will cause a 
text list of the user's personal messages to be displayed 
with another level of alphanumeric activation charac- 
ters, although not explicitly illustrated. The present 
invention can be expanded to incorporate any number 
of hierarchical levels within the scope thereof. 

It will be noted that the term "alphanumeric charac- 
ter" as used within the context of the present invention is 
not limited to letters and numbers, but includes any 
character that can be entered using a computer key- 
board. Thus, characters such as comma ".". period 
forward slash 7", and any other characters which may 
appear on conventional or special purpose keyboards 
are available for use within the scope of the invention. 

Although it is preferred to use only lower case letter 
and number characters and other characters that can 
be entered by depressing only one key (without using 
the shift key), the invention is not so limited, and the 
upper case character set which can be accessed using 
the shift key in combination with another key is also 
available. 

The implementation of the present invention will 
now be described with reference to FIGs. 5 to 12. 

FIG. 5 illustrates a computer system 100 incorpo- 
rating a GUI according to the present invention. The 
system 100 includes a random access memory (RAM) 
102 in which the GUI is stored in the form of a software 
program, and a processor 104 for executing the pro- 
gram code. The system 100 further includes a visual 
display unit or monitor 106 for providing a visual display 
of the GUI and associated application, a read-only 
memory (ROM) 108 for storing firmware, and an input- 
output (I/O) unit 1 10 for connection to a printer, modem, 
etc. 

The system 100 further includes a mass data stor- 
age 112 which can be any combination of suitable ele- 
ments such as a fixed (hard) magnetic disk drive, a 
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removable (floppy) disk drive, an optical (CD-ROM) 
drive, etc. Large application programs which incorpo- 
rate a GUI of the present invention may be stored in the 
storage 112, and blocks of the programs loaded into the 
RAM 102 for execution as required. 

User access to the system 100 is provided using an 
alphanumeric keyboard 114 which can be of conven- 
tional design. As shown, the keyboard 114 includes one 
or more label keys, such as a control (Ctl) key 1 1 4a and 
an alternate (Alt) key 1 14b, as well as a set of alphanu- 
meric character keys 114c. The system 100 may also 
include a pointing device such as a mouse (not shown). 

A functional block diagram of a GUI 120 according 
to the present invention is illustrated in FIG. 6. The GUI 
120 includes a display controller 122, a selection con- 
troller 124 and a launch controller 126 which are con- 
nected to the keyboard 1 14 for response to depression 
of keys. The display controller 122 and the selection 
controller 124 produce outputs which are displayed on 
the monitor or display 106. 

The display controller 122 superimposes and dis- 
plays characters corresponding to the character keys 
1 14c on the displayed graphical objects in response to 
depression of the label key (the Alt key). The selection 
controller 124 selects objects in response to depression 
of the character keys 1 14c corresponding to the charac- 
ters superimposed on the objects. 

The display controller 122 includes an object table 
122a including an entry for each graphical object, with 
each entry including an object name and a correspond- 
ing character. The display controller 122 further 
includes a display generator 122b which accesses the 
object table 122a using the object names as inputs, 
obtains the activation characters as outputs, and super- 
imposes and displays the characters on the objects. 

Each entry in the object table 122a further includes 
an object display position, and the display controller 1 22 
displays the characters at character display positions 
corresponding to the object display positions. 

Each entry in the object table 122a further includes 
an object type, and the display controller 122 displays 
the characters at the character display positions which 
are offset from the object display positions by predeter- 
mined distances and directions corresponding to the 
object types respectively. 

For example, the characters "G" to M M M in FIG. 3 are 
displayed near the upper left corners of the correspond- 
ing icon buttons, and are thereby offset from the centers 
of the buttons upwardly and leftwardly by a predeter- 
mined distance and direction. The characters n A" to "F n 
are offset from the centers of the text objects only in the 
leftward direction by a predetermined distance. 

A label (Alt) key controller 122c detects depression 
of the label (Alt) key and controls the display generator 
122b to access the object table 122a as described 
above. 

The selection controller 124 includes a key input 
controller 124a which produces characters correspond- 



ing to depressed character keys, and an object selector 
124b which accesses the object table 122a using the 
characters produced by the key input controller 124a as 
inputs, obtains the corresponding object names as out- 
5 puts, and select objects corresponding to the object 
names. 

The launch controller 126 launches operations 
(procedures, applications, etc.) corresponding to 
selected objects by accessing a table (not shown) which 

10 links the operations to the characters. 

The GUI 120 is loaded into the RAM 102 for execu- 
tion, and can be distributed into several program mod- 
ules. The present invention also includes a digital 
storage structure which can be, as illustrated in FIG. 7, 

15 the RAM 102 with the GUI 120 loaded therein. 

The present invention also includes at least one 
digital recording medium which stores the GUI 120 for 
shipping to customers. The medium can be, for exam- 
ple, one or more optical disks (CD-ROM), one or more 

20 magnetic disks (floppy disks), or a combination thereof. 
As viewed in FIG. 7, the GUI 120 has components 
distributed in an operating system 130 (e.g. UNIX), an 
application 132 such as e-mail, and a window manager 
134. Most of the functionality of the GU1 120 is prefera- 

25 bly implemented in the operating system 130 and the 
window manager 134 so that all application programs 
can utilize the invention. 

The operating system 130 includes an EVENT 
QUEUE 136 which detects and stores events such as 

30 key presses, and makes these events available to the 
rest of the system 100. The application 132 includes the 
object table 122a, and further includes a MAIN EVENT 
LOOP 138 and CHECK FOR ALT KEY PROCEDURE 
140 which in combination perform the functionality of 

35 the Alt key controller 122c and the key input controller 
124a. 

The application 132 further includes a GET LABEL 
PROCEDURE 142, a GET TYPE PROCEDURE 144, 
and a GET POSITION PROCEDURE 146, which, in 

40 combination with an ALT-LABEL PROCEDURE 148 and 
a DISPLAY MANAGER PROCEDURE 150 in the win- 
dow manager 134, provide the functionality of the dis- 
play generator 122b. A SELECT PROCEDURE 152 in 
the application 132 provides the functionality of the 

45 selection controller 124 and the launch controller 126. 

The operation of the MAIN EVENT LOOP 138 is 
illustrated in the form of a flowchart in FIG. 8. The loop 
138 continuously checks for depression of the Alt key 
114b by calling the CHECK FOR ALT KEY PROCE- 

so DURE 140. When the MAIN EVENT LOOP 138 detects 
depression of a character key after depression of the Alt 
key, it calls the SELECT PROCEDURE 152. The MAIN 
EVENT LOOP 1 38 also checks for other events that are 
required by the application 132, and loops back to the 

55 beginning. 

The CHECK FOR ALT KEY PROCEDURE 140 is 
illustrated in FIG. 9. The procedure 140 first checks to 
see if the Alt key was depressed by interrogating the 
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main event queue 1 36. If the Alt key was not depressed, 
the procedure 140 returns to the MAIN EVENT LOOP 
138. If the Alt key was depressed, the procedure 140 
causes the activation characters to superimposed on 
the corresponding graphical interface objects of the dis- 
play. 

The procedure 140 first checks to see if a current 
object is the last object. If so, the procedure 140 returns 
to the MAIN EVENT LOOP 138. If not, the procedure 
140 determines if the current object is being displayed 
(is visible). If not, the procedure 1 40 loops back to check 
the next object. If so, the procedure 140 calls the ALT 
LABEL PROCEDURE 148. The procedure 140 loops 
back until all objects have been processed. 

The ALT Lj^BEL PROCEDURE 148 is illustrated in 
FIG. 10. The procedure 148 sequentially calls the GET 
LABEL PROCEDURE 142, the GET TYPE PROCE- 
DURE 144, and the GET POSITION PROCEDURE 146 
which access the object table 122a with the name of the 
object as an input and obtain the label (corresponding 
character), object type and object position respectively 
as outputs. The procedure 148 then calls the DISPLAY 
MANAGER PROCEDURE 150 and returns to the 
CHECK FOR ALT KEY PROCEDURE 140. 

The DISPLAY MANAGER PROCEDURE 150 is 
illustrated in FIG. 11, and combines the oval shaped 
frame with the activation character. The procedure 150 
calculates the position at which to display the character 
by algebraically combining the display position of the 
corresponding object with the offset corresponding to 
the object type. The procedure 1 50 then adds images of 
the frame and character to the bitmap image of the dis- 
play in the RAM 102 and returns to the ALT LABEL 
ROUTINE 148. The combined bitmap image is dis- 
played by the system 100 in a normal manner. 

Upon detection of depression of a character key 
after depression of the Alt key, the main event loop 138 
calls the SELECT PROCEDURE 152 which is illus- 
trated in FIG. 12. The procedure 152 accesses the 
object table 122a with the character corresponding to 
the depressed key, and obtains the name of the corre- 
sponding graphical object. The procedure 152 then 
access a table (not shown) to obtain the operation (pro- 
cedure, application, etc.) corresponding to the selected 
object, and launches the operation. 

In summary, the present invention improves over 
prior art mnemonic character, keyboard accelerator and 
location cursor arrangements in that it does not require 
a separate text menu. The present arrangement is 
much faster than a location cursor system in that only 
two keystrokes are required to select a graphical object. 

The arrangement of the invention is especially 
desirable for persons suffering from afflictions such as 
carpal tunnel syndrome, because any graphical object 
can be selected using simple key presses. The inven- 
tion also enables quadriplegics to access graphical user 
interfaces from a keyboard using a single stick. 

Various modifications will become possible for 



those skilled in the art after receiving the teachings of 
the present disclosure without departing from the scope 
thereof. For example, although the present GUI has 
been described and illustrated as being implemented in 
5 software, the invention is not so limited, and the func- 
tionality of the GUI can alternatively be implemented in 
hardware in the form of, for example, an Application 
Specific Integrated Circuit (ASIC). 

io Claims 

1. A computer graphical user interface (120) config- 
ured to display and select graphical objects (52, 56, 
58, 62, 64, 68. 72, 74, 76) using a keyboard (114) 

is that has at least one label key (1 1 4b) and a plurality 
of character keys (1 14c). characterized by: 

a display controller (122) configured to super- 
impose and display characters (A-V, 1-7) corre- 
20 spending to the character keys (114c) on the 

Objects (52. 56, 58. 62, 64, 68, 72, 74. 76) 
respectively in response to depression of said 
at least one label key (1 14b); and 

25 a selection controller (1 24) configured to select 

objects (52, 56, 58. 62, 64. 68, 72, 74. 76) in 
response to depression of character keys 
(1 14c) corresponding to the characters (A-V. 1 - 
7) superimposed on the objects (52, 56, 58, 62, 

30 64, 68. 72, 74, 76) respectively. 

2. The interface of claim 1 , characterized that the dis- 
play controller (122) comprises: 

35 an object table (122a) including an entry for 

each object (52, 56, 58, 62, 64, 68, 72, 74. 76). 
each entry including an object name and a cor- 
responding character (A-V, 1-7); and 

40 a display generator (122b) configured to 

access the object table (122a) using the object 
names as inputs, obtain the characters (A-V, 1- 
7) as outputs, and superimpose and display the 
characters (A-V. 1 -7) on the objects (52, 56. 58, 

45 62, 64, 68, 72, 74, 76) respectively. 

3. The interface of claim 2, characterized in that each 
entry in the object table (122a) further includes an 
object display position ; and 

50 

the display controller (122) is configured to dis- 
play the characters (A-V, 1-7) at character dis- 
play positions corresponding to the object 
display positions. 

55 

4. The interface of claim 3 characterized in that: 

each entry in the object table (122a) further 
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includes an object type; and 
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the display controller (122) is configured to dis- 
play the characters (A-V, 1-7) at the character 
display positions which are offset from the s 
object display positions by predetermined dis- 
tances and directions corresponding to the 
object types respectively. 1 0. 

The interface of any of claims 2-4 characterized in 10 
that the display controller (122) further comprises a 
label key controller (122c) configured to detect 
depression of said at least one label key (144b) and 
control the display generator (122b) to access the 
object table (122a) using the object names as 15 
inputs, obtain the characters (A-V, 1-7) as outputs, 11. 
and superimpose and display the characters (A-V, 
1-7) on the objects (52, 56, 58, 62, 64, 68, 72, 74, 
76) respectively in response thereto. 



20 



The interface of any of claims 1-5 characterized in 
that: 

the display controller (122) comprises an object 
table (122a) including an entry for each object 
(52, 56, 58, 62, 64, 68, 72, 74, 76), each entry 
including an object name and a corresponding 
character (A-V, 1-7); and 



The interface of any of claim 1-8, characterized in 
that the display controller (122) is configured to 
superimpose and display the characters (A-V, 1-7) 
on the objects (52, 56, 58, 62, 64, 68, 72, 74, 76) 
only while said at least one label key (114b) 
remains depressed. 

The interface of any of claims 1-8, characterized in 
that the display controller (122) is configured to 
continue to superimpose and display the characters 
(A-V, 1-7) on the object (52, 56, 58, 62, 64, 68, 72, 
74, 76) after said at least one label key (1 14b) has 
been released and until a character key (1 14c) has 
been depressed. 

The interface of claim 10. characterized in that the 
display controller (122) is further configured to con- 
tinue to superimpose and display the characters (A- 
V. 1-7) on the objects (52, 56, 58, 62, 64, 68, 72, 74, 
76) until said at least one label key (1 14b) is again 
depressed. 



1 2. The interface of any of claims 1-11, characterized in 
that said at least one label key (114b) consists of 

25 one label key (1 1 4b). 

1 3. The interface of any of claims 1-11, characterized in 
that: 



the selection controller (1 24) comprises: 30 

a key input controller (124a) configured to 
produce characters (A-V, 1-7) correspond- 
ing to depressed character keys (1 14c); 

35 

an object selector (124b) configured to 
access the object table (122a) using the 
characters (A-V, 1 -7) produced by the key 
input controller (1 24a) as inputs, obtain the 
corresponding object names as outputs 40 
and select objects (52, 56, 58, 62, 64, 68, 
72, 74, 76) corresponding to the object 
names respectively. 

The interface of claim 6, characterized by a launch 45 
controller (126) configured to launch operations 
corresponding to selected objects (52. 56, 58, 62, 
64, 68, 72, 74, 76) respectively. 



said at least one label key consists of two label 
keys; and 

the display controller is configured to superim- 
pose and display characters corresponding to 
the character keys on the objects respectively 
in response to simultaneous depression of the 
two label keys. 

1 4. The interface of any of claims 1-13, characterized in 
that the display controller (122) is further configured 
to display frames (79) around the characters (A-V, 
1-7) respectively. 

15. The interface of claim 14, characterized in that the 
frames (79) have oval shapes. 



The interface of claim 7, characterized in that: 



1 6. The interface of any of claims 1-15, characterized in 
that the objects (52, 56, 58, 62, 64, 68, 72, 74, 76) 
comprise at least one of an icon button (56), a 
so scrolling text list (58), and a text entry field (72). 



the objects (52, 56, 58, 62, 64. 68, 72, 74, 76) 17. 

comprise a text entry field (72); and 

the launch controller (126) is configured to, 

when the text entry field (72) is selected, posi- ss 

tion a cursor (80) at an end of text in the text 

entry field (72). 



A computer implemented method for graphically 
displaying and selecting objects (52. 56, 58, 62, 64, 
68, 72, 74, 76) using a keyboard (114) that has at 
least one label key (1 14b), and a plurality of charac- 
ter keys (1 14c), characterized by the steps of: 

(a) superimposing and displaying characters 
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(A-V, 1-7) corresponding to the character keys 
(114c) on the objects (52, 56, 58. 62, 64, 68, 
72, 74, 76) respectively in response to depres- 
sion of said at least one label key (1 14b); and 

5 

(b) selecting objects (52, 56, 58, 62, 64, 68, 72, 
74, 76) in response to depression of character 
keys (114c) corresponding to characters (A-V, 
1-7) superimposed on the objects (52, 56, 58, 
62, 64, 68, 72, 74, 76) respectively. 10 

18. The method of claim 17, characterized in that step 
(a) comprises the substeps of: 

(a1) storing an entry for each object (52, 56, is 
58. 62, 64, 68, 72, 74, 76) in an entry table 
(122a). each entry including an object name 
and a corresponding character (A-V. 1 -7); 

(a2) accessing the object table (122a) using 20 
the object names as inputs; 

(a3) obtaining the characters (A-V, 1 -7) as out- 
puts; and 



(a4) superimposing and displaying the charac- 
ters (A-V, 1-7) obtained in step (a3) on the 
objects (52, 56. 58. 62, 64, 68, 72, 74, 76) 
respectively. 

19. The method of claim 18, characterized in that: 



25 



22. 



23. 



62, 64, 68. 72, 74, 76); 

a processor (104) configured to execute the 
program (132); 

a graphical display (106) configured to display 
the objects (52, 56. 58. 62, 64, 68. 72. 74. 76); 
and 

a keyboard (114) having at least one label key 
(1 14b) and a plurality of character keys (1 14c); 

characterized in that the graphical user 
interface (120) is an interface of any of claims 
1-16. 

A digital data storage structure (102) which stores a 
computer program (132) including a graphical user 
interface (120) characterized in that the graphical 
user interface is an interface of any of claims 1-16. 

The structure of claim 22, characterized by an elec- 
tronic memory (102) configured to store the compu- 
ter program (132). 



24. The structure of claim 23, characterized by at least 
one digital recording medium (102) configured to 
store the computer program (132). 



30 



step (a1) further comprises storing an object 
display position in each entry in the object table 
(122a) respectively; and 35 



step (a4) further includes displaying the char- 
acters (A-V, 1-7) at character display positions 
corresponding to the object display positions 
respectively. 40 



20. The method of claim 19, characterized in that: 



step (a1) further comprises storing an object 
type in each entry in the object table (122a) 45 
respectively; and 

step (a4) further includes displaying the char- 
acters (A-V, 1-7) at the character display posi- 
tions which are offset from the object display 
positions by predetermined distances and so 
directions corresponding to the object types 
respectively. 



21. A computer system (100), comprising: 

a memory (102) configured to store a program 
(132) including a graphical user interface (120) 
which displays selectable objects (52, 56, 58, 



EP 0 816 987 A2 



CO 



o 
m 



CM 
CN 



00 CO 
^ ^r- rO 
CN 



CN 



w 





Ld 




5. 








CO 




O 








Ld 




COI 




DIT 


1 1 1 


Ldl 


f— 

CO 




>- 


Ld 


CO 


CO 


1 


O 


1 


Q_ 


< 






O 


o 


Ol 






o 


Q i 


C£ 
1 — 


ct: 


< 


o 




Lu 


C£ 


_l 
1 i I 


Si 




is 




Q_ 




Ldl 




Cd 




Q 




< 




Ld 




cri 






■lll^ 



o 

Ld 

CD 
ZD 
CO 



o 
ct: 



Ld 
h- 
< 



o 

CN 



V 



o 



or 

ft 
5 



3 

LU 

CO 



o 
9e 

GO 



CO 
CN 



CL 
CL 
< 



2 
X 

o 



a: 
a. 
< 



o 

2 



ct: Ld L_ O Ld CO < 

I I I I I I I 

tibtititjbb 

<<<<<<< 



Q Ld 



< q. q: 

Ld Ld O 

cr ct: u_ 



o 

% O Ld 

2 b z > 

O Q Ld < 
O Ld CO CO 



h-H 

X 

o 



ct: 

Ld 

o 

CL 



o 

Ld 

~ 3 

CO 
ZD 
CO 




o 



SS E 

cc 

o o 

O 

CO 

Q CO 



O 
CO 



Ld 



si 



00 
CN 



L?) Ld 
C£ C£ 



ct: 
< 

ct: 
o 



Ld 
CO 
O 

it §2 

O Q Ld < 
O UJ GO 00 



CN 



CN 



rt 



EP 0 816 987 A2 




C\2 

• i— • 
fa 



EP 0 816 987 A2 




EP 0 816 987 A2 



O 

m 



CO 

m 



CD 
CO 



CN 
CO 



GO CO 

^ m 

CO 



O 



7 



UJ 

I— 

00 

>- 

00 



o 

z 
o 

C£ 
I— 

o 

UJ 

_J 

LU 





o 

Ld 

CD 
ZD 
00 



o 



o 



bJ 



o 

CO 



o 
z 

CO 



z 
< 



3 
CD 
a: 

CO 



a: 
o 



cr 

CO 



CO 
CO 



Q- 



Z 

o 



cr 
a. 
< 



X 

o 

CO 



o 
o 

CD 

00 
00 
L±J 
Ol 
Q 
Q 
< 



CM 

m 



UJ 



< 



00 

oi 
o 



> 



>- 

oc 
o 



UJ 

> 



o 



Ol 
LU 

o 

Q_ 



00 
UJ 
O 
Ol 
ZD 
O 
00 
UJ 

oi 



UJ 
00 
ZD 
O 
X 



Rl 



Ol 
UJ 
I — 
X 
UJ 



o 

00 
Ol 
UJ 
Q_ 



.CM 
00 



♦ 



EP 0 816 987 A2 



100 



1 14-C 



S 



114 





Q 


W 


E 


R 


T 


Y 












































































C1 


tri 


Alt 























114a^> Ml 



4b 



1 



108 



READ-ONLY 
MEMORY (ROM) 



112 



I 



DATA STORAGE 



^ — 




5 



102 



RANDOM ACCESS 
MEMORY (RAM) 



110 



— ► 



INPUT-OUTPUT 
(I/O) UNIT 



Fig.5 



138 












CHECK FOR ALT KEY 


I 




CHECK FOR OTHER KEYPRESSES 


1 




CHECK FOR OTHER EVENTS 







Fig.8 



EP 0 816 987 A2 




EP 0 816 987 A2 



* 



s 



102 



RANDOM ACCESS MEMORY (RAM) 



120 



OPERATING SYSTEM 



EVENT QUEUE 



130 



136 



APPLICATION 



132 



MAIN EVENT LOOP 



CHECK FOR ALT KEY PROCEDURE 
GET LABEL PROCEDURE 
GET TYPE PROCEDURE 
GET POSITION PROCEDURE 
SELECT PROCEDURE 
OBJECT TABLE 



^138 

^-140 
\_142 
^144 

\-146 
\-152 

-~^122a 



WINDOW MANAGER 



ALT— LABEL PROCEDURE 
DISPLAY MANAGER PROCEDURE 



-134 

148 
- 150 



Fig. 7 



EP 0 816 987 A2 




Fig. 9 



EP 0 816 987 A2 



148 




CALL GET LABEL 



I 



CALL GET TYPE 



I 



CALL GET POSITION 

I 



CALL DISPLAY MANAGER 




Fig. 10 



150 



(^^tartJ^) 

combine oval shape and 
activation character 



152 



I 



ADD TO BITMAP 
MEMORY IMAGE 



I 




Fig. 11 




GET 


OBJECT 




LAUNCH 


PROCEDURE 



J, 

Fig. 12 



